Natural Language Inference : from Textual Entailment to Conversation Entailment

NATURAL LANGUAGE INFERENCE: FROM TEXTUAL ENTAILMENT TO CONVERSATION ENTAILMENT By Chen Zhang Automatic inference from natural language is a critical yet challenging problem for many language-related applications. To improve the ability of natural language inference for computer systems, recent years have seen an increasing research effort on textual entailment. Given a piece of text and a hypothesis statement, the task of textual entailment is to predict whether the hypothesis can be inferred from the text. The studies on textual entailment have mainly focused on automated inference from archived news articles. As more data on human-human conversations become available, it is desirable for computer systems to automatically infer information from conversations, for example, knowledge about their participants. However, unlike news articles, conversations have many unique features, such as turn-taking, grounding, unique linguistic phenomena, and conversation implicature. As a result, the techniques developed for textual entailment are potentially insufficient for making inference from conversations. To address this problem, this thesis conducts an initial study to investigate conversation entailment: given a segment of conversation script, and a hypothesis statement, the goal is to predict whether the hypothesis can be inferred from the conversation segment. In this investigation, we first developed an approach based on dependency structures. This approach achieved 60.8% accuracy on textual entailment, based on the testing data of PASCAL RTE-3 Challenge. However, when applied to conversation entailment, it achieved an accuracy of 53.1%. To improve its performance on conversation entailment, we extended our models by incorporating additional linguistic features from conversation utterances and structural features from conversation discourse. Our enhanced models result in a prediction accuracy of 58.7% on the testing data, significantly above the baseline performance (p < 0.05). This thesis provides detailed descriptions about semantic representations, computational models, and their evaluations on conversation entailment.